INTRO COPYRIGHT horio shoichi 1995 INTRO NAME intro.txt - Introduction To Concache A Brief Overview Concache.exe is for "concurrent disk cache" that is so called "advance cache", which drives disks concurrently with DOS/user programs. The maximum concurrency it can perform is up to 25 disks, including one of two floppies, one of IDE disk, and up to maximum number of configurabe SCSI disk units under DOS. (The term IDE is for Integrated Device Interface. In this and the other texts in this package, how- ever, includes ST506 interface and ESDI for brevity.) Current version limits the maximum number to 16, since exceeding this limit sounds ridiculous. However, the ulti- mate limit of its concurrency is imposed only by DOS drive letter availability. Concache.exe allows true postwrite operations transparently for programs under DOS. The basic idea to attain concurrency is, by catching PC/AT type machine's BIOS int1590 and int1591 interrupts, to force switch from concache.exe to DOS/user program by the former and force back again to concache.exe by the latter. The SCSI hard disks are operated concurrently by the aid of ASPI (Advanced SCSI Programming Interface) command posting capability, which offers the chance to synchronize with disk operations exactly the same way as BIOS int1590/int1591 allow. Additional programs floppies.exe and ccdisk.exe are provided with concache.exe, the main cache program. The former is a mini-driver which does most of floppy i/o independent of DOS. The latter is the reentrant disk driver for SCSI disks, using ASPI. If a disk is neither of above, for example pre-ASPI SCSI disk, concurrency is lost there. On such disk, the interval the device made CPU idle is indiscernible from CPU overhead. Yet postwrite, although not concurently, is supported for even such disk. Concache.exe uses memory for caching disk data. Memory types to be used are 1) Raw protected (int15) memory, 2) Expanded memory conformable to EMS specification 4.0, 3) Extended memory block (EMB) defined by XMS 2.0 Concache 1.00 Last Update: 18 March 1995 1 INTRO COPYRIGHT horio shoichi 1995 INTRO specification. If only int15 memory is to be used no additional third party software is needed. If memory manager allows to configure, these three types of memory can be used freely mixed, totaling up to 64 mega- bytes. The amount of EMS or EMB used by concache.exe are changeable without disloading/reloading concache.exe from memory. (Int15 memory is hard to share with the other pro- grams. Changing its size on the fly is not considered.) Concache.exe resident memory requirements are usually around 20 kilobytes, increased depending on concurrency require- ments (stacks to keep things concurrent and sheer io buffers are unavoidably allocated at each startup.) Although concache.exe can be run on upper memory block (UMB), its large startup memory size (around 50kb) may not allow loading into UMB, even if more than 20 kb is available after many device drivers are loaded. To help this situation, there are two ways: 1) concache.exe can be loaded from config.sys command line as a device driver, giving an earlier chance to be loaded into UMB. 2) The option load_umb lets concache.exe try to find UMB of enough size for resident part. Prerequisites Concache.exe runs PC/AT class machine and after, equipped with INTEL 80386 CPU and the later, running MSDOS3.1 and the later or the equivalents. Installation If ROM BASIC is used then interrupt identification of pro- grams have to be adjusted using cctame.exe. Please refer to the document. Otherwise there is no installation procedure. Disable disk cache program currently used. Simply copy to convenient place and make calls, either from DOS config.sys, autoexec.bat, or command line, as appropriate. Quick Test Real programmers may skip to the next section. (Virtuals may better not.) Concache 1.00 Last Update: 18 March 1995 2 INTRO COPYRIGHT horio shoichi 1995 INTRO If your system is fairly recent one and you believe it is configured in just standard way, and you have read the Known Incompatibilities section below, and is not running the other disk cache program, and currently on command line from command.com, then you can test if your system runs concache.exe. (Sorry this sentence is so obscure. Although there is no 100 per cent guarantee, following procedure should work if DOS 5.0 or the later is used.) You should read the related documents to find out what you have done in this test. - First, go to the directory where concache.exe is copied and type simply "concache". You will see the message it is not loaded. This message should appear. - Next, type "concache x". Now concache.exe has allo- cated memory for its cache area if it doesn't complain. If it complains, try "concache e" and "concache p" in this order until you succeed. If all fails, there is no memory to be used for cache area, and your quick test has failed. You have to add memory or you must decrease some other program's memory usage. - You should have allocated all available memory of a type. See this by typing "concache hm", which means "help about memory usage". - If you succeeded in "concache "p", you can release the memory by typing "concache d" which means "delete resident concache.exe from memory". If you have done so, you might try different amount by typing "concache p1m", "concache p256k", "concache p100000", and so on, anything between zero and maximum value reported. - If you have succeeded in "concache x" or "concache e", you can try above experiment similarly. Try it later, substituting "x" or "e" with "p" as appropriate. For now type "concache xm" or "concache em", which reallocates zero megabyte (that is, zero byte) memory. In other words, it has released all allocated memory but concache.exe is still in memory. Again type "concache hm" and see it is using none. You are now be able to allocate as much memory as you like exactly like the case "concache 'p" but without doing "concache d". This is an advantage over "concache Concache 1.00 Last Update: 18 March 1995 3 INTRO COPYRIGHT horio shoichi 1995 INTRO p". - If you have allocated more than 256kb of memory, you will make an interesting test at this point. Prepare a spare floppy (it must be formatted, though) into floppy drive a:, and type "copy concache.exe a:" (or whatever file), and type "concache hd" immediately. You have told "help about drives" by typing "concache hd". You see floppy is being written, and you get the list of how devices are treated by concache.exe. You will observe a line "still XX dirty spots" which tells how many sectors remain to be written into the floppy a:. If you type "concache hd" again, then this line is probably unseen. By allocating larger memory and copying more files at once into floppy, and typing "concache hd" in succes- sion, you will observe the number of dirty spots decreasing toward zero. Chance to observe this line on hard disks are less unless a large copy is done using large memory. If you have succeeded up to this point, your remaining thing to do is to edit your autoexec.bat or config.sys file. How- ever, please read on. You have seen an advantage and a disadvantage of using concache.exe. You will feel comfortable to see that your computer responds quickly even if it is busy writing. But... While you DON'T HAVE TO wait for the complesion of writes to continue work, you MUST wait for it to pick the floppy out. There are many ways to await the complesion. The simplest is watching the access lamp. and the second is repeatedly typ- ing "concache hd" to make sure there is no dirty spots, as you have just learned. Another method is to type "concache r" to tell refresh drives. Then concache.exe watches the progress, and returns to command line when no dirty spots remains. Likewise when you shutdown your machine, there is a danger of remaining dirty spots. A recommended procedure is to have a shutdown procedure, like shut.bat, which contains at least "concache r" line (after the line which might cause the final writes to disks, of course). Remark: The author hates follow the piper approach. It Concache 1.00 Last Update: 18 March 1995 4 INTRO COPYRIGHT horio shoichi 1995 INTRO looks to him like a conspiracy to hide important things. In fact, if you read other text files you will find many impor- tant informations are missing in this subsection. This sub- section is written on the case best fortunate, to show although command line operands of concache.exe is compli- cated, its genuine nature is rather its simplicity. Actu- ally, all but two of his DOS machines are set up in this simplest way. Acceptance Test Since there can be many unnoticed incompatibilities, and they potentially cause the worst disasters, things may not be as simple as preceding subsection. In particular, following areas need thoroughly checked. - Memory managers. Options for "unique" features for a memory manager must be checked with special care. In general, simply using XMS or EMS would be well compatible. - Communication programs There are two considerations to be paid. One is if the CPU overhead concache.exe incurs, or the pattern of its CPU usage, can be tolerated by communications programs. The need to check this applies to serial communica- tions. The other is the possibility of loss of synchronization caused by mismatched handling of int2a (which concache.exe may be in part responsible for). This caution applies to the server side of so called "peer to peer" lans. - Multitaskers Most task switchers seem to work. Multitaskers need special care. - Disk compression and encryption programs These programs use proprietary driver schemes, which concache.exe doesn't know. - Third party block device drivers By definition, device drivers can do anything, and can be incompatible with concache.exe. However, well behav- ing device drivers can coexist. If int13 and int15 are used in predefined manner, these drivers will happily take advantage of concurrency. - File maintenance programs These programs can use int13 directly, which can cause Concache 1.00 Last Update: 18 March 1995 5 INTRO COPYRIGHT horio shoichi 1995 INTRO the mismatching of cached data and actual data in disks. If only int21, int25, int26 are used then they are generally safe. - OEM versions of DOS There may be differences from what concache.exe expects on MS/PC-DOS, mostly on io.sys/ibmbio.com. - Itchy spots Following is the list of current limitations of the author's test configurations. Main memories above 10mb for cache, More than one SCSI host adapters, More than two floppies, Other than DOS partitions on SCSI disks. Although the author believes there is no critical boun- dary conditions to cause troubles, any difficulty caused by exceeding above limits is definitely a bug. You should alert yourself you are crossing the border. An alert tester should proceed, if in any doubt, as follows. If needed, read concurrency mode to write after or write through mode. 1) Backup Make backup before conducting the tests. 2) If Ever Loadable Load concache.exe setting all drives in stop mode. 3) The Least Damage Put a floppy into concurrency mode. and try read and write on it. At the end, check the integrity of media with, for example, chkdsk program. 4) Add hard disks Put drives, one by one, into concurrency mode, and con- duct test on them. Integrity check is equally impor- tant. 5) Finally For at least a month after acceptance, more frequent backups are scheduled. Known Incompatibilities The programs are tested against various environments and have identified several incompatibilities, some solved, and some given up. A few command arguments try to solve Concache 1.00 Last Update: 18 March 1995 6 INTRO COPYRIGHT horio shoichi 1995 INTRO problems rather non-obvious ways. If you get into trouble, please give a look at eqanda.txt file. It describes the backgrounds of these troubles so you may find the way to proceed. Following are known incompatibilities: 1) Under NWDOS 7.0, writing to a floppy trashes its fat. To solve this, either use write through or floppies.exe. 2) Desqview 2.31 and Desqview/X 1.1 cause crash. Probably the family is incompatible as a whole. Use write through method. 3) QEMM 7.3 stealth feature causes hang or crash. Only write through works. 4) Windows 3.1 does not implement full EMS 4.0. Probably the 3.0 and 3.11 behave the same way. EMS memory is freed while under it. Also, SCSI devices should not be used in concurrency mode, takeover or not, where a vm (virtual machine) can possibly run background. The switching causes SCSI manager hang. For SCSI disks use write after mode or DOS device driver. 5) Server.exe of Novell Personal Netware maintains its internal int2a counts incorrectly. It causes hang or crash. Write after method solves the problem. The other TSRs that intercept int2a for synchronization with DOS have potentially the same problem. 6) Most block device redirectors such as disk compression and encryption programs are not cacheable. Load concache.exe before them in order to make cacheable. The only compatibles are Microsoft Dblspace and Stac Electronics Stacker 3.12 to date. These programs must be loaded before concache.exe. 7) Applications which use DOS "unused" stack space may be corrupted by concache.exe assumption of its being free at device driver calls. It uses rather larger stack space. Development Environment The primary development environment contains the following indispensable softwares to the author. He devotes a lot of Concache 1.00 Last Update: 18 March 1995 7 INTRO COPYRIGHT horio shoichi 1995 INTRO thanks to the authors of these programs. Borland C/C++ 4.0 Borland International MKS Toolkit 4.1b Mortice Kern Systems Inc. Periscope 5.40 References Of numerous books I referenced, following are particularly important in writing this software. IBM Corp. "Technical Reference Personal Computer AT" Adaptec "Advanced SCSI Programming Interface (ASPI)" Andrew Shulman, et al. "Undocumented DOS" eds 1, 2 Frank van Gilluwe "The Undocumented PC" Ralf Brown, Jim Kyle "PC Interrupts" eds 1, 2 Joeff Chappel "DOS Internals" Microsoft "MS-DOS Programmer's Reference" V5, 6 Trademarks Product names in this and other files in this package are (registered) trademarks of respective publisher's or of manufacturer's. Lastly But Most Important Enjoy the parallel world of DOS single tasking operating system ! Concache 1.00 Last Update: 18 March 1995 8